Papers by Mostafa Karimi Manesh
Advancing Persian LLM Evaluation (2025.findings-naacl)
Copied to clipboard
Sara Bourbour Hosseinbeigi, Behnam Rohani, Mostafa Masoudi, Mehrnoush Shamsfard, Zahra Saaberi, Mostafa Karimi Manesh, Mohammad Amin Abbasi
| Challenge: | Existing evaluation approaches for large language models in low-resource languages like Persian lack comprehensive frameworks, limiting their ability to assess models’ performance over a wide range of tasks requiring considerable cultural and contextual knowledge. |
| Approach: | They propose to provide two new benchmarks to assess models' performance over a wide range of tasks requiring considerable cultural and contextual knowledge. |
| Outcome: | The proposed benchmarks challenge the current state-of-the-art models’ abilities in a variety of Persian language comprehension tasks while reducing data contamination while providing an accurate assessment of Persian LLMs. |